This invention relates to a distributed file system logging method. In particular this invention relates to a distributed file system logging method that intercepts file system writes to log files.
In a mass virtual hosting Web cluster, a Web application can run as many instances on many Web Servers or virtual Web Servers in the Web cluster. Each instance of a Web application will write status data to at least one log file resulting in many log files in many places and a complicated developer review. The number of log files can easily become too large for quick review and some consolidation is required. Several solutions have been proposed to consolidate the log files.
U.S. Pat. No. 7,356,590, filed by the Visible Measures Corporation, discloses a distributed capture and aggregation of dynamic application usage information that uses an application tracking method to access and view aggregated log files. The application tracking method sits in an application layer and listens for events and forwards the events to an aggregated log file.
U.S. Patent Application No. 2002/0087949, filed by Golender, V, et al, discloses a system and method for software diagnostics using a combination of visual and dynamic tracing and comprises information gathering modules. Each information gathering module gathers user input and output and consolidates such input and output for diagnostics.
U.S. Pat. No. 6,298,386, filed by the EMC Corporation, discloses a network file server having a message collector queue for connection and connectionless oriented protocols. Connection messages are intercepted and collected in the collector queue.
All of the above solutions rely on changes to be made to the application running on the server so when a new application is loaded then changes must be made.
Another known method of consolidating the log files is to perform batch merging of the log files when the web application is idle. However, due to the interactive nature of developing/debugging web applications and following log files, developers need near real time access to the log files. If the Web application is not idle for long periods then batch merging is always out of date.
In normally secure applications, developers are not allowed interactive access to the web servers. Furthermore, if the Web cluster had a load balancing mechanism then it would be difficult to find which Web server a particular debugging session had used.
Network solutions where the log file is saved remotely in a consolidated network place tend to be unreliable and are not preferred since log files are required for audit purposes.
Applications that solve this problem by logging to a database need to understand the large number of tables and database connections that would be needed in a mass virtual hosting application such as this prevent this approach.
It is also possible for Web servers to pipe access logs into processes but this is not the case for error logs.
U.S. Patent Application No. 2006/025373, filed by the EMC Corporation, discloses backing up selected files of a computer system by using a mirroring driver attached to file system driver to create back-up copies of data files concurrently with changes to data files. However, this disclosure does not address the problem of consolidating data logs.